Skip to content

feat(config): per-operation reasoning_effort override (#2998) - #3043

Merged
nicoloboschi merged 1 commit into
mainfrom
fix/per-operation-reasoning-effort-2998
Jul 29, 2026
Merged

feat(config): per-operation reasoning_effort override (#2998)#3043
nicoloboschi merged 1 commit into
mainfrom
fix/per-operation-reasoning-effort-2998

Conversation

@nicoloboschi

Copy link
Copy Markdown
Collaborator

Summary

reasoning_effort was the only LLM request setting with no per-operation override. Retain, reflect and consolidation each already accept their own PROVIDER, API_KEY, MODEL, BASE_URL, MAX_CONCURRENT, MAX_RETRIES, INITIAL_BACKOFF, MAX_BACKOFF, TIMEOUT and LITELLMROUTER_CONFIG — but all four operation configs read the single global HINDSIGHT_API_LLM_REASONING_EFFORT.

When one operation requires a specific value, every other operation is forced to take it too. Concretely: some OpenAI reasoning models reject function tools unless reasoning_effort is exactly "none". Reflect is a tool-calling search loop, so pointing it at such a model requires none — which then lands on retain and consolidation as well, silently degrading their generation quality (the reported case: retain emitting facts in Chinese from an English transcript).

Closes #2998.

Change

Adds REASONING_EFFORT to the existing per-operation set, following the established fallback pattern used by every neighbouring field:

HINDSIGHT_API_RETAIN_LLM_REASONING_EFFORT         -> falls back to HINDSIGHT_API_LLM_REASONING_EFFORT
HINDSIGHT_API_REFLECT_LLM_REASONING_EFFORT        -> falls back to HINDSIGHT_API_LLM_REASONING_EFFORT
HINDSIGHT_API_CONSOLIDATION_LLM_REASONING_EFFORT  -> falls back to HINDSIGHT_API_LLM_REASONING_EFFORT

An operator can now set reflect to none (satisfying the tool-call constraint) while retain keeps low.

  • config.py: three ENV_* constants, three str | None dataclass fields, three from_env() parses (empty/unset → None).
  • memory_engine.py: the three per-operation LLM configs read config.<op>_llm_reasoning_effort or config.llm_reasoning_effort. The global default config is unchanged.
  • docs: new rows in configuration.md (and its skill mirror).

Tests

TestPerOperationReasoningEffort in test_per_operation_llm_config.py:

  • per-operation overrides are parsed from env, global left untouched;
  • unset per-op fields stay None (fall back at runtime);
  • MemoryEngine threads each override into its LLM config, and an unset operation inherits the global value.

Notes

  • The per-operation override env vars are intentionally not enumerated in .env.example — matching the existing precedent (none of the ~30 other per-op override vars are listed there; the file documents the prefix-fallback pattern once and lists only the global HINDSIGHT_API_LLM_REASONING_EFFORT).

reasoning_effort was the only LLM request setting without a per-operation
override: retain, reflect and consolidation all read the single global
HINDSIGHT_API_LLM_REASONING_EFFORT. When one operation requires a specific
value (e.g. reflect needs "none" for OpenAI reasoning models that reject
function tools otherwise), that value is forced onto the others, silently
degrading their generation quality.

Add REASONING_EFFORT to the existing per-operation set, following the
established fallback pattern:

  HINDSIGHT_API_RETAIN_LLM_REASONING_EFFORT
  HINDSIGHT_API_REFLECT_LLM_REASONING_EFFORT
  HINDSIGHT_API_CONSOLIDATION_LLM_REASONING_EFFORT

Each falls back to HINDSIGHT_API_LLM_REASONING_EFFORT when unset.
@nicoloboschi
nicoloboschi merged commit b1a0ef5 into main Jul 29, 2026
103 checks passed
@nicoloboschi
nicoloboschi deleted the fix/per-operation-reasoning-effort-2998 branch July 29, 2026 12:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

reasoning_effort has no per-operation override, so a model constraint on one operation degrades the others

1 participant